The present invention relates to a technology on a vehicle surroundings monitoring device in which images taken with a camera capturing the surroundings of a vehicle are transformed to provide a synthesized image showing the situation around the vehicle.
In recent years, to help drivers drive safely, vehicle surroundings monitoring devices for monitoring the surroundings of a vehicle with a camera have been increasingly put into practical use.
In one type of conventional vehicle surroundings monitoring device, a camera is placed so as to capture a blind spot for the driver, such as an area behind the vehicle. Image taken with the camera are transformed by a predetermined fixed method, to be displayed on a display. For example, in a technique disclosed in Japanese Laid-Open Patent Publication No. 03-99952 (literature 1), a plurality of cameras are placed on a car as shown in FIG. 20A. Images of the surroundings of the car taken with the cameras are transformed, to produce an image viewed from top with the car in the center as shown in FIG. 20B, and the resultant image is presented to a display mounted in the car.
In another type, the position of an obstacle in an image taken with a camera placed on a vehicle is computed. The image is transformed according to the computed position of the obstacle, to be displayed on a display. For example, in a technique disclosed in Japanese Laid-Open Patent Publication No. 06-333200 (literature 2), the position of an obstacle in an image is computed using two images taken at slightly different times when the vehicle is moving. The images are transformed using the computed position of the obstacle, to obtain a synthesized image viewed from side as shown in FIG. 21A.
By use of the monitoring devices described above, the driver can check a blind spot, of which direct observation is not possible from the driver's seat, with a display, and can easily grasp how the obstacle is apart from the vehicle with an image viewed from top or side. In this way, the driver can drive more safely.
However, the conventional techniques described above have the following problems.
In the case of transformation of a camera image by a predetermined fixed method, no consideration is given to the shape of an obstacle in the image. Therefore, the shape of the obstacle is distorted in a synthesized image, and this makes the synthesized image unnatural. For example, in the literature 1, a camera image is transformed on the assumption that any object in the image exists in the same plane, that is, in the plane of road surface, for example. Therefore, as shown in FIG. 20B, while the road surface is displayed correctly, an object that is not in the plane of road surface, such as an obstacle, is displayed as a distorted unnatural image.
In the case of transformation of an image according to the position of an obstacle computed from images, a natural synthesized image with no distortion is produced when the position of the obstacle is correctly computed. However, if the computed position includes errors greatly, the obstacle may possibly be displayed at a wrong position, or the obstacle may not be displayed in a synthesized image. For example, in the literature 2, the position of an obstacle is computed by extracting the motion of a luminance edge from two images taken at slightly different times. In the actual drive environment, however, there often occur such cases that the computed position includes errors greatly and that the position of an obstacle is undetectable at all. In the case that the computed position includes errors greatly, the obstacle may be displayed at a position farther than the actual position, or the obstacle may not be shown in the synthesized image. This may mislead the driver and thus prevent the driver from driving safely.